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ABSTRACT 


This study examined the Naval Facilities Engineering 
Command's current use of construction contractor performance 
evaluations (SF 1420) from the viewpoint of accepted control 
and measurement theory. Surveys of field contract adminis- 
trators, supervisory civil engineers, field contract spe- 
Clialists, and construction contractors were completed to 
assess their use of and views about the evaluations. 

The study concluded that: (1) NAVFAC lacks standards of 
performance to describe the distinction between satisfac- 
tory, outstanding, and unsatisfactory performances (2) Con- 
tractors are not generally aware of the evaluation process; 
(3) Evaluations are not used to provide contractors feed- 
back; (4) Evaluators are not well trained; (5) Evaluations 
are not fully utilized; (6) The data base of evaluation 
information is inadequate. 

The study recommends that: (1) NAVFAC issue a policy 
statement to contractors to clarify the evaluations uses, 
standards, and performance elements; (2) interim evaluations 
be issued to provide contractors feedback; (3) contractors 
receive copies of all their evaluations; (4) evaluators 
receive uniform training on completion of evaluations; (5) 


SF 1420 be modified to allow for a more specific evaluation. 
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Ll. MNTRODUGCIICH 


The purpose of this study is to examine the Naval 
Facilities Engineering Command's (NAVFAC) current use of 
construction contractor performance evaluations (Standard 
Form 1420). That current use will then be analyzed using 
accepted theory about control systems, measurements, and 
performance appraisals. Based on the analysis some recom- 
mended improvements to NAVFAC's contractor performance 
evaluation system will be made. Appendix A defines some of 


the key words used throughout this study. 


A. RESEARCH QUESTIONS 

The primary research question is: how can NAVFAC make 
better use of the construction contractor performance evalu- 
ations? These cacenaeey research questions will also be 
addressed: 


1. How does NAVFAC currently use the performance 
evaluations? 


2. What does the Federal Acquisition Regulation (FAR) say 
about the contractor performance evaluations? 


3. What amount of attention do NAVFAC's managers place on 
the evaluations? 


4. Do the contractors have a formal system for evaluating 
their subcontractors? 


5. What do contractors think about the performance evalu- 
ation process? 


6. What alternative uses are there for the evaluations? 


7. What percentage of contracts result in outstanding, 
satisfactory, and unsatisfactory evaluations. 
B. METHODOLOGY AND LITERATURE REVIEW 
Mail and telephone surveys were used to determine the 
current uses of and attitudes about contractor performance 
evaluations. Different mail surveys were sent to: 
1. NAVFAC's Assistant Resident Officers-in-Charge of Con- 
struction (AROICC's) who directly administer construc- 


meron contracts. 


2. Construction contractors who have previously done work 
for NAVFAC. 


eee supervisory civil Sees who manage NAVFAC's field 
offices. 


Field contract specialists were surveyed by telephone. 
Literature on control and measurement systems was 
reviewed and provided a body of theory which was used to 
Breil ze the research data. The FAR, Department of Defense 
FAR Supplemental (DFARS), and NAVFAC's Contracting Manual 
(P-68) were used to develop background on the requirement 


for issuing contractor performance evaluations. 


C. LIMITATIONS 

A search for literature about other contractor perform- 
ance evaluation systems produced no relevant information for 
this study. Some data located at NAVFAC's Engineering Field 
Divisions (EFD) could not be provided due to the heavy 
seasonal workload demands. The survey population sizes were 


relatively small. The limitations on the use of data 


generated from these small populations are discussed in 


Chapter IV. 


De 


SUMMARY OF FINDINGS AND RECOMMENDATIONS 


This writer concluded that NAVFAC could significantly 


improve its use of the contractor performance evaluations. 


These recommendations for improvements are made: 


ile 


Tssue a policy statement describing to contractors the 
purpose for the evaluation system. 


Issue contractors interim evaluations mid-way through 
the Comerace. 


Provide contractors copies of their final evaluations. 


Provide AROICC's training on how to complete the 
evaluations. 


Increase management emphasis on the evaluation 
process. 


Provide meaningful rewards to contractors who are 
rated outstanding. 


Publicize the times evaluations are successfully used 
to deny contract awards to contractors with unsatis- 
factory performance records. 


Improve the access to and use of contractor perform- 
ance evaluation information. 


Improve the Standard Form 1420. 


ORGANIZATION 

The rest of this study consists of: 

Chapter IJII--Background information about NAVFAC's 
contracting environment and the regulatory requirements 
for contractor performance evaluations. 

Chapter) Pil =econtrol, measurement, and performance 


appraisal theory, and three models of possible uses for 
the evaluations. 
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Chapter IV--The research data summary. 


Chapter V--An analysis of contractor performance evalua- 
tions as a control mechanism. 


Chapter VI--Conclusions and recommendations 


Supporting appendices. 
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Il. BAGCKGEGRRE 


This chapter provides background information about: 


1. NAVFAC's construction mission and contractama 
environment 


2. NAVFAC's organization 


3. The Federal Acquisition Regulation (FAR) requirements 
for pre-award surveys and performance evaluations 


4. Standard Form 1420, Performance Evaluation- 
Construction Contract 


5. The Small Business Administration (SBA) role in con- 
tractor responsibility determinations 

A. NAVFAC BACKGROUND 
The Navy Facilities Engineering Command maintains the 
U.S. Navy shore facilities throughout the world. NAVFAC 
employs more than 25,000 people to ensure its mission is 
properly performed. NAVFAC augments its forces by contract- 
ing out more than $2 billion worth of constructionmed 
civilian firms each year. NAVFAC's goal is to ensure each 
contract is completed safely, on time, within cost and at 

satisfactory quality. 
To accomplish that goal NAVFAC iS organized on the 


levels shown in the diagram below: 


2 


Headquarters 
6 Engineering Field Divisions 


Field Offices 


Construction Contractors 


NAVFAC Headquarters provides overall guidance, makes 
policy decisions, and provides staff support to the 
Engineering Field Divisions (EFD). Each EFD awards con- 
struction contracts, monitors contract progress, and assists 
field offices in their contract administrative efforts. The 
field offices make some contract awards and administer the 
construction contracts. Each field office is managed by a 
supervisory civil engineer. Contract specialists award the 
contracts and provide expertise in contractual matters. The 
contract administration is performed by civilian contract 
administrators and by Civil Engineer Corps Officers (Assis- 
tant Resident Officers-in-Charge of Construction, AROICC's). 
Technical inspection of construction work is performed by 
construction specialists. 

Almost all NAVFAC construction contracts are awarded 
using a publicly advertised sealed bid process. Under this 


process NAVFAC advertises its prospective contracts using an 
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invitation for soader Contractors compete for the contract 
award by submitting sealed firm-fixed-price bids. The 
contract is awarded to the lowest responsive responsible 
bidder. To be responsive a bidder must meet all the condi- 
tions of the bid invitation. To be determined responsible a 
bidder must be able to demonstrate the capacity and capabil- 
ity to accomplish the contract work. NAVFAC has Little con- 
trol over who its low bidders are. It does have some 
control in determining whether a contractor is deemed 
responsible. Responsibility determinations are made during 
a pre-award survey. The next section describes the regula- 
tory requirements concerning pre-award surveys and contrac- 


tor performance evaluations. 


B. REGULATORY REQUIREMENTS 
1. Pre-Award Surveys 
During é Pre-Award survey, a contract specialist 
will examine the apparent low bidders qualifications to 
determine whether or not that contractor is responsible. To 
be determined responsible a prospective contractor must meet 


several criteria, including "Have a satisfactory performance 


record." (FAR, 1984) pan. 9. le4ic)) Contracts shall be 
awarded to responsible prospective contractors only." (FAR, 
1984, par. 9.103(a)) The determination of a contractor's 


responsibility is important because: 
The award of a contract to a supplier based on lowest 


evaluated price alone can be false economy if there is 
subsequent default, late deliveries, ope other 
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unsatisfactory performance resulting in additional 
contractual or administrative costs. While it is impor- 
tant that Government purchases be made at the lowest 
price, this does not require an award to a supplier solely 
because that supplier submits the lowest offer. A pros- 
pective contractor must affirmatively demonstrate its 
responsibility, including when necessary, the 
responsibility of its proposed subcontractors. (FAR, 
984. par. 9.103(c)) 


2. Contractor Performance Evaluations 
A major source of information about a contractor's 


past performance is their contractor performance evaluations 


Maem prior contracts. Evaluation of contractor performance 
is a special aspect of contracting for construction. The 
FAR says: 


(ob) Preparation of Performance Evaluation Reports: 

(1) The contracting activity shall evaluate contractor 

performance and prepare a performance report for each con- 
struction contract of 


(As) $500,000 or more; 

(ii) $100,000 or more, if any element of per- 
formance was either unsatisfactory or 
outstanding; 


(111) More than $10,000, if the contract was ter- 
minated for default; or 


(iv) $100,000 or more, if the contract was ter- 
minated fom the convenience of the 
Government. 


(2) The report shall be prepared at the time of final 
acceptance of the work, at the time of contract termina- 
tion, or at other times, as appropriate, in accordance 
with agency procedures. Ordinarily, the evaluating offi- 
Cial who prepares the report should be the person respon- 
sible for monitoring contract performance. 

(3) If the evaluating official concludes that a con- 
tractor's overall performance was unsatisfactory, the con- 
tractor shall be advised in writing that a report of 
unsatisfactory performance is being prepared and the basis 
for the report. If the contractor submits any written 
comments, the evaluating official shall include them in 
the report, resolve any alleged factual discrepancies, and 
make appropriate changes in the report. 

(4) The head of the contracting activity shall 
establish procedures which ensure that fully qualified 
personnel prepare and review performance reports. 


ibs, 


(b) Review of performance reports. Each performance 
report shall be reviewed to ensure that it is accurate and 
fair. The reviewing official should have knowledge of the 
contractor's performance and should normally be at an 
organizational level above that of the evaluating 
OfLVeTaln 

(c) Distribution and use of performance reports. (1) 
Each performance report shall be distributed in accordance 
with agency procedures. One copy shall be included in the 


contract file. The contracting activity shall retain the 
report for at least six years after the date of the 
report. 


(2) Before making a determination of responsibility in 
accordance with Subpart 9.1, the contracting officer may 
consider performance reports in accordance with agency 
instructions. (FAR, 19847 "par... 360.200) 


The Department of Defense FAR Supplement mandates 
that "Performance evaluations of construction contractors 
shall be used in making responsibility determinations." 
(DFARS, 1984, pare 36.201 (e NAVFAC's P-68 offers this 
guidance about contractor performance evaluations: 


PERFORMANCE REPORT. Contractor performance reports 
are valuable contract records and should be prepared by 
qualified personnel in a careful and conscientious man- 
ner. These reports must be based on factual rather than 
Subjective data. These reports frequently form the basis 
for the selection of contractors for the accomplishment of 
critical work. They are essential in findings of nonre- 
sponsibility for contractors that have done prior work for 
NAVFACENGCOM. OICC's are responsible for assuring evalua- 
tion reports are promptly and accurately completed and 
distributed. Unless the contractor correctly points out 
factual errors, performance evaluation reports are not to 
be revised merely to meet contractor objections; rather, 
the contractor's comments are to be attached to the 
evaluation report. (P-68, 1985, par. 6-206) 


C. STANDARD FORM 1420 
Standard Form 1420, PERFORMANCE EVALUATION-CONSTRUCTION 
CONTRACTS is used for reporting contractor performance. A 


blank SF 1420 is included as Appendix B. The form allows 
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space for general contract data including: contractor name, 
complexity and description of work, contract start and 
finish dates, contract award amount, contract change orders, 
and the extent of subcontracting. Part II allows the rater 
to evaluate six performance elements (Quality of Work, Time- 
liness of Performance, Effectiveness of Management, Compli- 
ance with Labor Standards, Compliance with Safety Standards, 
and Overall Performance) as either Outstanding, Satisfac- 
tory, or Unsatisfactory. Any marks of outstanding or 
unsatisfactory must be supported by a narrative on the back 
of the evaluation. The evaluations are normally filled out 


by the AROICC upon completion of the contract work. 


D. THE ROLE OF THE SMALL BUSINESS ADMINISTRATION 

Statistics provided by NAVFAC's Small Business Liaison 
indicate that small business concerns receive approximately 
80% of NAVFAC's contracts (dollar value) each year. 


Whenever a prospective contractor is a small business, the 
Small Business Act vests in the Small Business Administra- 
tion sole authority to make a determination of nonrespon- 
Sibility. An affirmative responsibility determination by 
SBA is issued via the medium of a Certificate of Competen- 
eve (COC). If SBA declines for any reason to issue a 
certificate of competency, the prospective contractor 
Shall be rejected as not responsible. (P-68, 1985, par. 
4-407.4(b)) 


The Small Business Administration (SBA) has this role in 
determining a contractor's responsibility: 


(a) Upon determining and documenting that a 
responsive small business lacks certain element of respon- 
sibility (including, but not limited to, competency, capa- 
bility, capacity, credit, integrity, perseverance, and 
tenacity), the contracting officer shall-- 


Ly 


(1) Withhold contract award (see 19.602-3), and 

(2) Refer the matter to the cognizant SBA 
Regional Office in accordance with agency procedures 

(c) The referral shall consist of-- 

(1) A notice that a small business concern has 
been determined to be nonresponsible, specifying the ele- 
ments of responsibility the contracting officer found 
lacking, and 

(2) A copy of the solicitation, drawings and 
specifications, preaward survey findings, pertinent tech- 
nical and financial information, abstract of bids (if 
available), and any other pertinent information that sup- 
ports the contracting officer's determination. (FAR, 
1984, par. 19 s6e2-1) 


Once the referral has been made to the SBA, the 
following procedures are followed: 


(a) Within 15 business days (or a longer period 
agreed to by the SBA and the contracting agency) after 
receiving a notice that a small business concern lacks 
certain elements of responsibility, the SBA will take the 
following actions: 

(1) Inform the small business concern of the 
contracting officer's determination and offer it an oppor- 
tunity to apply to the SBA for a certificate of competency 
(COG)}2 

(2) Upon timely receipt of the application and 
required documentation, send an SBA team to visit the con- 
cern to investigate it only for the specific elements of 
responsibility that the agency notice specified as 
lacking, and to make recommendations to the SBA Regional 
Administrator. 

(3) If the Regional Administrator plans to issue 
or recommend issuance of a COC, provide advance notice of 
the proposed action to the contracting officer together 


with a brief statement of the reasons for it. If the con- 
tracting officer disagrees with the proposal, resolve the 
disagreement as provided in 19.602-3. (FAR, 1985) 3ipaume 
19.602-2) 


E. SUMMARY 

This chapter described NAVFAC's mission, its organiza- 
tion, and its contracting procedures. The chapter also 
reviewed the regulatory requirements for pre-award surveys 


and contractor performance evaluations. Finally, the 
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chapter discussed the Standard Form 1420 and the Small 
Business Administration's role in determining a contractor's 
competency. The next chapter examines accepted theories 
about control and measurement systems and develops three 
models of potential uses for the contractor performance 
evaluations. The theory and models will be used later 


during the analysis of research data. 


Eo 


III. CONTROL AND MEASUREMENT THEORY 


The contractor performance evaluations provide a measure 
of the contractor's performance Foneyan conerace, The 
contractor performance evaluation process is intended to 
control the quality of construction contractors who rece. 
NAVFAC contracts. This chapter examines accepted theories 
of measurement, control, and evaluation relevant to NAVFAC's 


use of contractor performance evaluations. 


A. “CONTROL favor. 

Management control is defined as, ". . . The process 
through which managers assure that actual activities conform 
to planned activities. .. . The control process .. . en- 
ables managers to detect deviations from the plan in time to 
take corrective action before it is too late." (Stoner, 
1982, p. 592) This definition implies that even though one 
is "in control," actual performance can vary from planned. 
A manager exerts control when he takes action to eliminate 
variations from plan. 

All control systems have some common characteristics 


- - any control system has at least these four 
components: 


1. An observation device that detects or observes and 
measures or describes the activities or other phenomena 
being controlled. The term for this component may be 
observer, detector, or sensor. 
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2. An assessing device that evaluates the performance of 
an activity or organization, usually relative to some 
standard or expectation of what should be, and identifies 
out-of-control activities and conditions. The term for 
this component is evaluator assessor, or selector. 


3. A behavior modification device for altering or chang- 
ing performance if the need for doing so is indicated. 
This component may be called a director, modifier, or 
effector. 
4. A means of transmitting information among the other 
devices. This component's term is communication network. 
(Anthony, Deardon, Beford, 1984, p. 8, underlines added 
for emphasis) 

This model portrays the functioning of a control system 


according to Anthony. 


CONTROL DEVICE 


DETECTOR EVALUATOR SEF FECTOR 











CONTROLLED ENTITY 


The arrows in the model constitute the communication 
network. In this control system the detector, selector, and 
effector functions could all be performed by the same 
person, or each could be performed by different people. 
Within a control system: 

Individuals typically occupy one of three organiza- 
tional positions with respect to each information and 


control system they interact with: 


1. The measured and controlled position. This position 
involves being measured by the system. 
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2. The system maintenance position. Their primary task 
is seeing that the decision makers have the information 
they need. 

3. The decision maker position. This position involves 
receiving the information from the control system in order 
to make decisions. (Lawler and Rhode, 1976, p. 8) 

The control system affects the behaviors of people in 
each position in different ways. Usually, the behavior of 
greatest concern is that of the person being measured. 
Those who design control systems should consider’ the 
system's effects on behavior to ensure dysfunctional 
behaviors are not produced. 

All information and control systems have these charac- 
teristics in common: 


1. They all have some similar structural characteristics. 


a. They all collect, store, and transmit information 
in the form of abstract measures of reality. 


b. The collected abstract information is stored and 
transmitted in a specific form and with specific 
frequency. 


c. The summarized information is distributed Meow 
specific, usually pre-determined, group of people. 


2. All information tries to accomplish the same thing-- 
influence behavior. The crucial aspect of any control 
system is its influence on behavior. (Lawler and 
Rhode, 1976, (pp-5>—o) 

Given that control systems have some basic similarities, 
one can ask, what qualities make one control system better 
than others? How are good control systems developed is 


another good question. A good place to start is deciding 


the purpose of the control systen. 
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Some measure performance to provide management with the 
information they need to control employee's’ present 
behavior. Others provide top management with the inform- 
ation they need for long-range planning. And a third 
group provides ongoing feedback to employees about their 
job performance. (Lawler and Rhode, 1976, p. 38) 
B. CONTRACTOR PERFORMANCE EVALUATIONS AS A CONTROL SYSTEM 
Lawler and Rhode's’' previous quote described three 
possible uses for control systems. Let us now examine how 
NAVFAC's Contractor Performance Evaluation system could 
fulfill each of those three uses. 
1. Use I: Award Decisions 
Providing top management with information for long- 
range planning is currently the actual purpose of NAVFAC's 
contractor performance evaluation system. The evaluations 
provide top management with information which is used to 


make decisions about whether contractors receive future con- 


tract awards. This model describes the control system: 


CONTROLS Y STEM 












AROICC | OICC/EFD 
| 
DETECTOR EVALUATOR EFFECTOR 


Observed 
Performance 


Award 
Decisions 


CONTRACTOR 
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In this model the AROICC functions as the detector 
and evaluator. The OICC or EFD acts as the effector because 


the action of interest Pee tmertueureweont caer na. ace 
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2. Use IT: Controlling Ongoing Contractor Performance 


If NAVFAC issued interim performance evaluations to 
contractors, then their behavior on current contracts could 
be affected. This model depicts the evaluation system 


functioning as a control of a Centractor's current behaviem 


CONTROL SYSTEM 






AROICC 


DETECTOR——EVALUATOR——-9EFFECTOR 





Observed Performance 


Performance Evaluation 


CONTRACTOR 


Notice in this model that the AROICC performs the 
detector, evaluator, and effector functions by himself. The 
AROICC is the effector because the action of interest is 
controlling the ongoing contractor performance. 

3. Use III: Controlling ArOICC Performance 

Lawler and Rhodes third use, providing ongoing feed- 

back to employees about their job performance, could be 


satisfied under NAVFAC's current evaluation system. 
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Specifically, top management could control the AROICC's use 
of contractor performance evaluations by rating the AROICC's 
performance on: (1) the preparation of contractor evalua- 
tions, and (2) the usefulness of contractor performance 
evaluations as a source of information prior to administer- 
mie a contract. The AROICC's performance should improve 
because preparing a better contractor performance evaluation 
will require increased documentation and understanding of 
the contractor's performance. Reviewing the contractor's 
past performance evaluations prior to administering a new 
contract will alert the AROICC to that contractor's previous 
problems. This two-tiered model depicts that control possi- 


bility within the NAVFAC organizational structure: 


CONTROL SYSTEM 
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DETECTOR—_> EVALUATOR ——_EFFECTOR 
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Evaluation Rating 
Preparation 
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AROICC OECC 
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In this model, on the upper tier the OICC acts as 
the detector, evaluator, and effector. The OICC is the 
effector because the action of interest is controlling the 
AROICC's performance. 

The three models just presented include the four 
basic characteristics of control systems described by 
Anthony. The evaluation system also meets the criteria des- 
cribed earlier by Lawler and Rhode, because: 


- The evaluations eollect- store, and transmit 
information. 


- The information is stored in a specific format (SF 1420) 
and is collected with specific frequency. 


- The summarized information is distributed to a specific 
group of people (OICC, EFD, Army Corps of Engineers). 


- The evaluations purpose is to assist in determining the 
responsibility of contractors. If one assumes contrac- 
tors want to be determined responsible, then the evalua- 
tions can be assumed to have an influence on their 
behavior. 


C. CONSIDERATIONS IN DEVELOPING AND EVALUATING CONTROL 
SYSTEMS 
A control system provides the user with information in 
the form of a measure. The following items represent some 
characteristics which make information useful to its users: 
1. Usefulness for Decision Making--The characteristics of 
information that make it a deSirable commodity can be 


viewed as a hierarchy of qualities, with usefulness 
for decision making of most importance (FASB, 1980, p. 


4037). There are many user-Specific factors to con- 
Sider when deciding what information is most useful 
for decision making. These factors include: ", . . 


the decisions to be made, the methods of decision 
making to be used, and the information already posses- 
sed or obtainable from other sources." (FASB, 19c0e 
Dea ded) 
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Understandability--"Information cannot be useful to 
decision makers who cannot understand it, even though 
it may otherwise be relevant to a decision and be 


reliable." (FASB, 1980, p. 4037) 
Relevance--"Does the*measure contain information per- 
tinent to the decision at hand?. .. . Looking for- 


ward, relevance implies predictive ability; .. . 
Looking backward, relevance implies feedback value." 
(Moses, 1986, p. 9) "Relevant information is capable 
of making a difference in a decision by helping users 
to form predictions about the outcomes of past, 
present, and future events or to confirm or correct 
prior expectations. Information can make a difference 
to decisions by improving decision makers' capacities 
to predict or by providing feedback on earlier expec- 
tations. Without a knowledge of the past, the basis 
for a prediction will usually be lacking." (FASB, 
1980, p. 4038) 


Timeliness--"If information is not available when it 


1s needed or becomes available so long after the 
reported events that it has no value for future 
action, it lacks relevance and is of little or no 
use." (FASB, 1980, p. 4038) "Timeliness does not 
cause relevance, but lack of timeliness can make an 
otherwise useful measure irrelevant." (Moses, 1986, 


onl) 


Reliability--"To what degree do repeated measurements 

of the same attribute vary?" (Euske, 1984, p. 86) 
"Reliability connotes stability, consistency, objec- 
tivity, dependability, and absence of bias. In short 
reliability is the relative absence of errors of 
measurement in the measurement instrument." (Moses, 
UI86, Dp. 5) 


Requisite Variety--"Do the measures make sufficiently 
fine distinctions among objects? A given decision 
Situation may reguire a particular kind of separation 
or differentiation among objects. If a measurement 
process is to provide useful information for the deci- 
Sion, it should result in measures that capture the 
required distinctions." (Moses, 1986, p. 12) 


Completeness--"A crucial aspect of any control system 
is how completely and inclusively it measures the 
behaviors that need to be performed by a job holder. 
vo: If measures are incomplete, a person will be 
motivated to perform only a portion of the behavior 
needed for organizational effectiveness." (Lawler and 
Rhode, 1976, p. 42) 
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8. Valence--"To what degree are the measurements tied to 
the organizational reward system? .. . Individuals 
act to satisfy personal needs. To the extent that 
some needs are satisfied by the organizational reward 
system, and to the extent that rewards are linked to 
specific méasurements, behavior to influence the 
measurements will be motivated." (Moses, 1986, p. 16) 


9. Controllability--"Can the person measured influence 
the events or objects that are represented by the 
measurements? Without control, individuals cannot 
feel responsible for outcomes or feel satisfaction in 
outcomes, and consequently, will not be intrinsically 


motivated to perform. Without control individuals 
will see no link between his or her behavior and 
potential rewards, and consequently will not be 
extrinsically motivated." (Moses, 1986, pp. 16-17) 
10. Objectivity--"Some systems use objective measures 
while others use subjective ones. .. . Research 
shows that the more objective measures are, the more 
likely they are to motivate behavior. ... It is 


only logical that if individuals don't understand how 
the measure operates because it is highly subjective, 
they will see little connection between their behavior 
and any reward based on the measure. .. . Another 
reason objectivity of measures is important has to do 
with the climate of mistrust present in most organiza- 
tions. When employees do not believe they will be 
evaluated fairly, they do not believe that good per- 
formance on their part will lead to rewards, and as a 
result motivation is low." (Lawler and Rhode, 1976, 
pp. 42-47) 


D> SOLER CONSIDERATIONS FOR DESIGNING A CONTROL SYSTEM 

Once the purpose for a system is made clear, and all the 
user-specific needs are identified, the system designer can 
proceed. These other factors should be considered when 
developing a control system. Following each listed factor 
below is a description of that factor, and a discussion of 


what makes each factor a desirable feature of a control 


system. 


Exe. 


1. Nature of Standards--The research and theory on 
extrinsic motivation make one point quite clear: 
Employees are not motivated to reach goals that they 
don't feel they can achieve. .. . The research on 
goal setting (e.g., Locke, 1968) suggests that indi- 
viduals tend to achieve easy goals but not “exceed 
them. The result is mediocre performance. . . To set 
goals at the proper level of difficulty, individuals 
who have information on where the standards should be 
set must be involved. Further, the individuals whose 
performance iS being measured must be aware of the 
information that was used to set the standards so they 
will realize that the standards are reasonable. 


2. Source of Discrimination--As with standard setting, it 
1S crucial who acts as discriminator. .. . Whoever 
it is must have two attributes: the knowledge to make 
the comparisons and the trust of the person being 


measured. 

3. Pattern of Communication--Control systems vary widely 
in terms of who receives information about deviations 
from the standard. ... For extrinsic motivation to 


be present, the person who has the power to give 
rewards must receive the information about the results 
of the discrimination. This iS a precondition to 
rewards: being allocated on the basis of performance 
which in turn is a precondition to the perception that 
rewards are based upon performance. However, for this 
perception to exist, .. . it is desirable to have the 
person whose performance is being measured, and other 
employees who are doing similar work, receive the 
communication. 


4. Speed and Frequency of Communication--Control systems 
differ in how quickly they report the results of their 


measurement process. .. . AS a general rule, commun- 
ications about performance measures are most effective 
in producing extrinsic motivation when they are fast 
and frequent. When the communications about perform- 
ance are delayed, it is impossible to closely tie 
rewards to the actual performance of the person and 
this has the effect of reducing the perceived rela- 
tionship between performance and rewards. It also 
creates doubts among employees about the validity and 
usefulness of the performance measurement data. 
(Lawler and Rhode, 1976, pp. 42-43) 


The theory on performance appraisals offers this advice: 


Today's employees want more from an organization than 
just their salary. They want to know how they are 
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performing in their jobs. .. . People do not mind being 
evaluated if the appraisal system is fair and comprehen- 
Sive to them. (Cohen and Jaffee, 1982, p. 94) 


Wells says an effective and defensible performance 
appraisal system should have these qualities: 


1. Understandable Standards--Every employee should under- 
stand the standards against which he or she will be 
evaluated. .. . They should have prior knowledge of 
what is expected. 


2. Knowledgeable Appraisers--Another problem with similar 
implications, may arise when immediate superiors do 
not possess sufficient technical knowledge or 
expertise in an area of specialization to reasonably 
judge the results produced by others. 


3. Appraisers Should Be Well Trained--There are a number 
of reasons why it is critical that those conducting 


the appraisals should be well trained to guarantee 
consistency, . . . Consistency in administration is 
integrally linked to the defensibility of the system 
in the event of a legal challenge. 


4. Communication of Policy and Purpose--The purposes and 
uses of appraisals should be clearly stated in the 


policy manual, as well as in the employee handbook. 

Fully informing all employees of appraisal policy 
and purposes will minimize uncertainty and potential 
resistance. Employees understanding will increase the 
potential for positive results from appraisals. 
(Wells, 1982, pp. 777-781) 

The items Wells stresses are consistent with Lawler and 
Rhodes considerations for control systems. The theories and 
models presented in this chapter will be used in Chapter V 
during the analysis of NAVFAC's current use of performance 
evaluations. The next chapter presents the data generated 


during the research about NAVFAC's current use of contractor 


performance evaluations. 
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IV. RESEARCH DATA 


A. OVERVIEW 
This chapter describes the research methods used to 

examine NAVFAC's current use of Contractor Performance 
evaluations. The research data produced is also presented. 
The following sources of data were used: 

1. Mail survey of AROICC's. 

2. Mail survey of construction contractors. 

3. Telephone survey of field office contract specialists. 


4. Mail requests for information from EFD points of 
eoncact. 


5. Mail survey of field office Supervisory Civil 
Engineers. 


6. Small Business Administration sources. 

The mail and telephone surveys were prepared using 
Dillmans' Total Design Method (Dillman, 1982). The Total 
Design Method is a book which provides comprehensive 
instructions on how to prepare successful mail and telephone 
Surveys. Among other things, the Total Design Method: (il) 
helped reduce evaluation bias, (2) gave tips for increasing 
the survey response rate, and (3) provided examples of 
properly phrased survey questions. The purpose for, back- 
ground about, and survey results for each data source are 


presented below. 
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B.  AROICC SURVEY 
1. Purpose 
A mail survey of AROICC's was used to determine: 


- AROICC opinions about the effectiveness and usefulness 
of the contractor performance evaluations. 


- The amount of training AROICC's have in completing an 
evaluation. 


= AROTECEe perceptions about management's emphasis-= on 
evaluations. 


- AROICC opinions on the use of interim evaluations. 
2. Background 
Before the AROICC survey form was finalized, 8 
"pilot" surveys were sent to AROICC's to determine if the 
responses would generate acceptable data. All 8 surveys 
were returned. A review of the responses resulted in the 
elimination of one question, and minor rewording of two 
others. The final survey (Appendix C) was sent to 60 
AROLTCE sce The AROICC's surveyed were chosen from NAVFAC's 
P-1, Navy Civil Engineer Corps Directory. Only LTJG's and 
above with more than 1 year experience aS an AROICC were 
-SuEeyeee 54 of the 60 (90%) surveys were returned. The 
data summary below includes the 8 pilot surveys, for a final 
response from 62 of 68 (91%) AROICC's. 
3. AROICC Survey Results 
The AROICC survey questions and summarized responses 


are presented below. 
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Question 1. Approximately how many satisfactory, outstand- 
ing, and unsatisfactory contractor performance evaluations 
have you written during your career aS a contract 
administrator? 


Totals: 1361° (86%) --Satisfactory 
119 (7.6%) --Outstanding 
100 (6.4%) --Unsatisfactory 
1580 
The average respondent has filled out 25.4 evaluations 
each. 


Question 2. Does your EFD have an instruction governing 
contractor performance evaluations? (All EFD's do have 
instructions.) 


45 of 62 (64%)--YES 
16 of 62 (26%)--NO 
1 of 62--Did not answer 


Question 3. If answer to question 2 was yes, how much help 
does the instruction provide when you are filling out an 
evaluation? 


1 of 45--A Big Help X 3 points = 03 
22 of 45--Some Help X 2 points = 44 
16 of 45--A Little Help xX 1 point = 16 
4 of 45--No Help X O points = _0 
2 of 45--Did not answer 65 points/43 
respondents = 
1 Ghow 


The weighted average response of 1.51 falls mid-way between 
a little and some help. 


Question 4. Have you ever received any training on how to 
complete the performance evaluation’ form? 


23 of 62 (38.1%) --YES 
39 of 62 (61.9%) --NO 


Pee Yo, which of these statements most accurately 
describes the level of your training? 


4--Formal Classroom training 

5--Briefed during AROICC staff meeting 

11--OJT by Contract Specialist or another AROICC 
12--Given a copy of an old evaluation as a guide 
SZ 


*--Some of the respondents listed more than one 
type of training 
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Question 5. Do you ever review a contractor's record of 
past performance prior to the start of a new contract? 


23 of 62 (38.1%)--YES 
39 of 62 (61.9%)--NO 


If yes, approximately how often do you do so? 


12 do so more than 50% of the time 
11 do so less than S02 sofeene cine 


Question 6. Do you know where you could find a file of all 
of a particular contractor's past performance evaluations? 


25 of 62 (40%)--YES 
37 of 62 (60%)--NO 


Question 7. On a scale of 1 to 10, how effective do you 
think the existing performance evaluation system is at 
improving the overall performance of NAVFAC contractors? (1 
not effective to 10 very effective) 


not effective very effective 
scale--0 il 2 3 4 5 6 7 8 9 1K 
tenes 1h 11 asi: 20 q) 3 2 3 QO. 0 1 
responses 
The average was 3.03. A trimmed average (omit the 


lowest and highest response) was 2.96. 73% of the responses 
were 3 or less. 


Question 8. Have you ever written an interim evaluation on 
a contractor mid-way through a project? 


15 of 62 (24%)--YES 
47 of 62 (76%)--NO 


If yes, what effect does the interim evaluation have ona 
contractor's performance? 


10 of 15 said there was a noticeable improvement in the 
contractor's performance. 


Question 9. Do you think interim evaluations would help you 
to positively influence a contractor's behavior? 


41 of 62 (67%)--YES 


18 of 62 (29%) --NO 
3 of 62 ( 4%)--Did not answer 
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Why? The YESES typically said: 


a. 


196 


C. 


Gi 


Contractor will adjust behavior to please the 
NAVY 

Farly feedback will prove useful to the 
GoOncracclor 

Will provide an evaluation on file if the 
AROICC is transferred 

COnuracctors are concerned about their 
reputations 


The NO's typically said: 


a. 


Question 10. 


Contractors are only motivated by money 


Which of these statements most accurately 


describes how much emphasis your office management places on 
the performance evaluations? 


18 of 62 
19 of 62 
14 of 62 


5 of 62 


aeOL 62 
iF of 62 


Question 11. 


evaluate our 


)--Evaluations are stressed frequently 
)--Evaluations are stressed occasionally 
)--Evaluations are stressed if contractor 
performance unsatisfactory 
( 8%)--Evaluations are stressed when overdue to 
the EFD 
)--Evaluations are never stressed 
%)--Did not answer 


0 oP ol? 


(29 
(31 
(22 


What comments do you have about the way we 
contractors? Please include any suggestions 


for improving the evaluation process. 


This is a summary of the comments received. 


Frequenc 


Is: 


alse. 


Comment 


The evaluations are not effective because 
NAVFAC issues subsequent awards to contractors 
who've gotten unsatisfactory evaluations 


The evaluation forms need more performance and 
rating categories to allow for a more detailed 
evaluation of the contractor's performance 


The EFD's need to issue periodic summaries of 
contractor performance for use by = field 
offices 

Unsatisfactory evaluations are avoided because 
they take too much time, effort, and documen- 
tation to support 


The evaluations are too subjective 


es 


A policy should be established to mandate 
denial of awards after a certain number of 
unsatisfactory evals 


We need meaningful rewards for outstanding 
COUCEAGEOE. 


The Small Business Administration rarely rules 
against a contractor in a Certificate or 
Competency case 


Interim evaluations should be sent EO 
contractors 


AROICC's should be better trained in filling 
out evaluations 


Time constraints affect the quality of 
evaluations 


The narrative should be written in bullet 
statements 


Need some criteria for translating separate 
ratings on each item into an overall rating 


A contractor's performance is affected by the 
quality of the project design 


Copies of the evaluations should be sent to 
the contractors' bonding companies 


CONTRACTOR SURVEY 


5 Purpose 


The co 


COontracreor 
performanc 


ContCraccor 
process an 


ntractor survey was used to determine: 


opinions about the quality of NAVFAC's 
e evaluations. 


opinions about the performance evaluation 
d how it might be improved. 


Contractor systems for evaluating subcontractors 


Contractor 


opinions about receiving interim evaluations. 
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2s Background 


Time constraints prevented the use of a "pilot" 
survey. Contractor mailing lists with contractor names, 
addresses, and points of contact were requested from each 
EFD. Contractors on the lists were to have regularly per- 
formed contracts within the EFD. 4 of 6 EFD's provided the 


Gontractor lists, but only 2 of 4 included names of 


contractor points of contact. Using those lists, surveys 
were sent to 80 contractors. 32 of the 80 (40%) returned 


the surveys, but 13 did not complete them because they had 
never seen nor received a performance evaluation from 
NAVFAC. Of the 19 who did complete the surveys, 10 contrac- 
tors had seen or received an evaluation, and 9 had not. A 
copy of the contractor survey is included as Appendix D. 
3. Contractor Survey Results 

The contractor survey questions and summarized 
responses are presented below. 
Question 1. On a scale of 1 (no value) to 10 (high value), 


how much value do you assign to the performance evaluation 
rating you receive on a project? 


no value high value 
scale--1 2 3 4 5 6 7 8 =) 10 
# of --4 0 0 0 3 0 a 4 3 4 
responses 


The overall average was 6.6. The average for those who have 
seen an evaluation before was 7.9. 63% of the responses 
were 7 or above. 
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Question 2. How useful to you think interim evaluations 
would be in providing you feedback about your performance on 
a COnesace, 


of 19 (47%)--Very Useful 

of 19 (21%)--Fairly Useful 
of 19 (16%) --Useful 

of 19 (11%)--Of Little Use 
of 19 ( 5%)--Did Not Answer 


mm WwW £ O 


Of the 10 who've seen an evaluation before, 9 of 10 said 
fairly or very useful. 


Question 3. Would you like to see NAVFAC issue interim 
performance evaluations on its contracts? 


13 of 19 (68%)--YES 
6 of 19 (32%) --NO 


Why? The yes's said: 


a. It would eliminate any surprises in the final 
evaluation 
b. It would improve AROICC/contractor communications 


c. Contractors need feedback to know if work satisfies 
the government 

dad. Contractors can correct mistakes before it is too 
late 


The no's said: 
a. AROICC's are not qualified to make the evaluations 
b. Contractors already know how good their performance 


is. 
Question - Do you have a system for evaluating your 
subcontractors? 


None of the respondents have a formal evaluation system. 
All make basic subjective judgments about the subcontrac- 
tor performance. Those who don't perform well are not 
used on future jobs. 


Question 5. Would you find it useful to receive a copy of 
your performance evaluation for each contract you perform? 


18 of 19 (95%) --YES 
1 of 19 ( 5%)--NO 


Comments: These general comments were made by contractors 
who were surveyed: 
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Frequency Comment 


2 Evaluations are improperly used by AROICC's as 
a bargaining tool at contract close out 


2 Evaluations are too subjective 

2 Evaluations are poorly documented 

a Contractors should also evaluate the AROICC's 
al An appeal process 1s needed to mediate 


disputed evaluations 
1 Evaluations don, & matter because peor 


contractors still receive contract awards. 


D. CONTRACT SPECIALIST SURVEY 
1. Purpose 
A telephone survey of contract specialists in 20 of 
NAVFAC's field contract offices was used to determine: 


- How performance evaluations were used in support of the 
pre-award survey process. 


- If the evaluations provided contract specialists enough 
information to determine contractor responsibility. 


- Contract specialists attitudes about the level of 
management emphasis on performance evaluations. 


- Contract specialists' use of the EFD performance 
evaluation files. 


Zoe background 
Survey questions were asked uniformly to all 20 
respondents, and answers were consistently recorded. 5 Ob 
the 20 respondents ene not capable of answering some of the 
questions because of their offices' unique organizational 
structure. This information is used with the caveats that: 


(1) the survey population is statistically quite small, and 


os, 


(2) the size and volume of contracts administered in each 
surveyed office was not determined. A copy of the survey is 


included as Appendix E. 


3. Contract Specialist Survey Results 


The contract specialist survey questions and summar- 
1zed responses are presented below. 


Question 1. What percent of the time do you use a 
contractor's past performance evaluations to assist you 
during the Pre-Award Survey process? 


1 of 20 ( 5%)--Use the evaluations 100% of the time 

8 of 20 (40%)--Use the evaluations if the contractor 
has done work in their office before 

11 of 20 (55%)--Use the evaluations less than 10% of the 
time. They prefer to rely on phone 
calls to contractor provided references 
for information about ae contractor's 
past performance. 


Question 2. What percent of the time do you discuss the 
pre-award survey results with the AROICC assigned to the 
project? 


1 of 20 ( 5%)--The AROICC does the pre-award survey 

4 of 20 (20%)--Notify the AROICC 90% of the time 

11 of 20 (55%)--Notify the AROICC if the contractor's 
past performance is less than 
satisfactory 


4 of 20 (20%)--Do not notify the AROICC at all 


Question 3. Other than your own files, do you use any other 
sources of contractor performance evaluations? 


14 of 20 (70%)--Use no other source 
5 of 20 (25%)--Use their EFD's file less than 5% of the 


time 
1 of 20 ( 5%)--Does not even keep its own file of 
evaluations 
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Question 4. Do the performance evaluations give you enough 
information to do the pre-award surveys properly? 


8 of 20 (40%)--Said the information was adequate 

7 of 20 (35%)--Said the information on the evaluations 
was too general, and that more specific 
comments about contractor performance 
should be made 

5 of 20 (25%)--Provided an inadequate response 


Question 5. How much management emphasis is placed on 
performance evaluations within your office? 


of 20 (20%)--Strong management emphasis 

of 20 (35%)--A fair amount of management emphasis 
of 20 (25%)--A little management emphasis 

of 20 (20%)--Provided an inadequate response 


mm Os] 


Question 6. In the past year how many contractors did your 
office withhold awards from because of an unsatisfactory 
record of past performance? 


12 of 20 (60%) --Zero 
meOLr820 (25%)—-One 
yeone 20 (10%) —-—Four 
1 of 20 ( 5%)--Six 


E. EFD INFORMATION 

The following information was gathered from the EFD 
points of contact in response to mail and telephone 
requests. 1 of the 6 EFD's chose not to provide any 
information. 


1. All 5 EFD's have instructions governing the use’ and 
preparation of contractor performance evaluations. 


2. 2 of 5 EFD instructions encourage the use of interim 
evaluations if the contractor's performance is 
unsatisfactory. ; 

3. All EFD's keep past performance evaluations separated 


DyacOntractor:. 
4. Pacific Division augments the SF1420 with its own 


"Supplemental Information Sheet" (Appendix F). This 
sheet allows AROICC's to rate contractors on more 
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specific items. The sheet also has two extra ratings, 
marginal and exceptional. 


5. Pacific Division keeps an evaluation record form 
(Appendix G) on each contractor. That form summarizes 
a contractor's performance evaluations from many 
contracts. 

6. Northern Division is trying to implement a computer 
data base which will create a separate file of evalua- 


tion data for each contractor. The data base will be 
accessible to all of Northern Divisions field offices, 


F. SUPERVISORY CIVIL ENGINEER SURVEY 


i Purpose and Background 


A mail survey of 10 supervisory civil engineers was 


conducted to assess their attitudes about contractor per- 


formance evaluations. 7 responses were received (70%). A 
copy of the survey is included as Appendix H. Several of 


the survey questions were not well prepared and their 
responses were unsuitable for use. This information is used 
with the caveat that the sample population is statistically 
very small. 
2. Supervisory Civil Engineer Survey Results 

The supervisory civil engineer survey questions and 
summarized responses are presented below. Those questions 
which were inadequately prepared are not included here. 


Question 3. Do you provide your AROICC's any training on 
how to complete the performance evaluation form? 


3 of 7 said YES 

3 Of 7 Saldato 

1 of 7 does all evaluations himself, therefore no 
training needed 
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Question 4. Does your office keep separate performance 
evaluation files on each contractor? 


5 of 7 said NO 
2 of 7 said YES 


Question 6. Does your office write interim evaluations on 
contractors midway through a report? 


5 of 7 said NO 
2 of 7 said YES 


Question 7. Do you think NAVFAC should issue interim evalu- 
eens on all of its construction contracts? 


6 of 7 said NO--They fear the field office staff already 
has more work than it can comfortably 
handle 

fot: 7 Said YES 


Question 9. If you had access using the office computer to 
a file listing all of a contractor's past performance 
evaluations, would you use it to get a feel for how a pros- 
pective contractor of yours usually performs? 


6 of 7 said YES 
1 of 7 said NO 


Question 10. Do you think we should give our contractors a 
copy of their performance evaluations at the end of a 
Gonmtract? 


5 of 7 said YES 
2 of 7 said NO 


Question 11 asked for comments on how NAVFAC could improve 
its evaluation system. These responses were received: 


Frequency Comment 


2 The evaluations should include more specific 
items, and have more rating categories 


2 EvValilatlonc mhave a newrertreect ©on contractors 
because the poor performers continue to get 
new contract awards 


al: The excessive amount of documentation needed 
Ze) Support an unsatisfactory rating is 
burdensome 
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1 The form should require an explanation of a 
contractor's satisfactory performance also 


il Evaluations are given superficial treatment by 


AROICC's because it takes too much time to 
prepare a good evaluation 


G. SMALL BUSINESS ADMINISTRATION DATA 
1. Purpose and Background 
When small businesses are involved the Small 
Business Administration has the authority to override a 
NAVFAC decision of contractor non-responsibility. That 
override occurs when a Certificate of Competency is issued. 
2 of the AROICC's responding to the survey said the SBA 
rarely rules against contractors on COC decisions. The data 
presented below comes from the General Accounting Office 
report #RCED-86-120BR dated April 1986 and titled "Small 
Business Administration-Status, Operations, and Views of the 
Certificate of Competency Program." The report highlights 
actual uses of the contractor performance evaluations within 
the SBA's Coc system. 
2. SBA Data 
All data presented below comes from the above cited 
GAO report. The data represents all types of contracts 


reviewed by the SBA including some construction contracts. 
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a. This table provides statistics on the cOC activity for 





VOSI—-19385. 
Fiscal Refer- Contractor COC Applica- Direct 
Year rals Applica- approv- e1LOn Awards 
to SBA tions als denials without 
aeucOCG 
eS 1 2052 880 489 B22 i 
1982 PA sy bi 7) 998 495 409 FS 
19'S 3 2949 LO7 a 556 S93 20 
1984 3072 1099 547 378 Zo 
1985 4223 inepey 884 540 _54 
Totals 15733 5700 2977 2042 128 


The numbers of approvals, direct awards, and denials do not 
equal the number of applications because of such things as 
applications withdrawn, referrals withdrawn, and procure- 
ments withdrawn. 


Based on the total figures for the five years, there were 
10,033 contractors who did not apply for coOC's, and another 
2042 who were denied COC's. Those 12,075 contractors repre- 
sent 76% of the total referrals to the SBA. Thus 76% of 
contractors referred to the SBA do not receive the contract 
award. 


b. The GAO report included a survey of 279 cases. The 
survey looked at the processing time for those cases. The 
FAR allows the SBA 15 days to process a case, or a longer 
time if agreed to by the referring contracting agency. The 


results of the survey showed: 


Number of Cases 
Workdays Number Percent 
15 or less 128 44.6 
16-20 84 29.3 
Bi-—25 32 ie aes 
26-35 aly, 5.9 
36 or more melo Onis 
Total 279 100 


Though only 44.6% of the cases were completed within the 
allowed 15 days, 85% were completed within 25 days. 


c. The SBA's contractor review procedures are generally 
consistent with DOD's and GSA's review procedures. 
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d. No statutory or regulatory criteria exist to guide SBA 
in evaluating prior performance when making COC decisions. 
In addition, SBA has not established specific written 
criteria to make such determinations. The SBA evaluators 
must rely on professional judgment much like the AROICC's 
do. 
e. Of 287 cases sampled, 109 were referred to the SBA 
because of the contractors poor prior performance. Of those 
109, SBA agreed with the referring agency in 51 cases, or 
about 47%, and denied the Coc. 
H. SUMMARY 

This chapter introduced data from several sources about 
NAVFAC's current use of the contractor performance evalua- 
tions. The next chapter provides an analysis of that data 


from the viewpoint of accepted control and measurement 


theory from Chapter III. 
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V. ANALYSIS OF CONTRACTOR EVALUATIONS AS A CONTROL MECHANISM 


Chapter III presented some accepted control and measure- 


ment theory as well as three possible uses for the contrac- 


tor performance evaluations. Chapter IV presented data 


about NAVFAC's current uses of the contractor performance 


evaluations. This chapter provides an analysis of those 


current 


uses from the viewpoint of the accepted theories. 


The analysis focuses on the three possible uses for the 


evaluations from Chapter III. 







Peo L[--AWARD DECISIONS 

The FAR, DFARS, and NAVFAC P-68 each cite only one 
purpose for the contractor performance evaluations; to 
provide information about contractor past performance during 
the pre-award survey. This control model reproduced from 
Chapter III depicts the control system. 

CONTROL syvoTElM 
AROICC 
DETECTOR——> EVALUATOR 
observes award 
performance decisions 





CONTRACTOR 
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2. Analysis 

When analyzed from the viewpoint of accepted control 
theory, NAVFAC's current use of the contractor performance 
evaluation system for making contract award decisions is 
weak in the following areas. 

a. standards 

Lawler and Rhode, and Wells said control and 

appraisal systems should have well-defined standards. The 
standards should be understood, and should not be too easy 
or too difficult to attara The FAR, DOD FAR Supplement, 
and NAVFAC P-68 all are silent about standards for contrac- 
tor performance. No guidance exists which defines what con- 
stitutes satisfactory, disattersoceme or outstanding 
performance. Instead, each AROICC is left to exercise his 
own professional judgment when evaluating a contractor's 
performance. 

b. Usefulness For Decision Making 

This characteristic of information is most 

important. Despite its importance, 11 of 20 contract 
specialists use the evaluations less than 10% of the time 
when doing pre-award surveys, and 14 of 20 do not use other 
available sources of evaluations. 5 of 7 supervisory civil 
engineers said their offices don't keep separate files of 
performance evaluations. These facts indicate NAVFAC could 
improve its use of the contractor performance evaluations in 


the contract award decision making process. 
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The following association of responses by the 
contract specialists provides some insight into the useful- 
ness of the performance Bra lamin: The responses to 
question 1 (What * of the time do you use the evaluations 
during the pre-award survey process?) were matched against 
the responses to question 6 (In the past year how many con- 
tractors did your office withhold awards from because of an 
unsatisfactory record of past performance?) These results 
show that offices which used the evaluations more often were 


able to support more contract award denials: 


Question 6--Number of contractors denied awards last year 
0 ab Z 4 6 

Question 1--What % of: Less than 10%-- 8 3 0 0 0 
the time do you use 
evaluations during More than 10%-- 4 2 0) 2 1 
pre-award surveys? 

The offices who use the performance evaluations 
more often during Liew Pp eemcdiicdhCeatinvey = nOCcess ELoported 4 
higher incidence of denying contractors awards. 

c. Communication 

Wells says employees should be told the purpose 
and uses of performance appraisals to minimize uncertainty 
and resistance. The results of the contractor survey 
clearly indicate the contractors are unaware that the evalu- 
ation system even exists (22 of 32 had never received an 


evaluation). Lawler and Rhode say those being evaluated 


should receive the evaluations. NAVFAC currently notifies 
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contractors of their performance evaluation only if the per- 
formance is unsatieenerent (6.4% of the contracts). 
a. Traaining 
Wells says that appraisers should be well- 

trained. Only 38% of AROICC's surveyed said they'd been 
trained in completing evaluations (most of that training was 
Informa lye Without common training NAVFAC is left with 
AROICC's who hold a wide variety of ideas about: 

- The intent of each rating category 

- The amount of detail needed to support an evaluation 


- The kinds of documentation needed to ‘support an 
evaluation 


- The standards of performance for various ratings 
This wide variety of ideas reduces the relia- 
bility of the evaluation system. Recall that reliability 
connotes stability, consistency, objectivity, dependability, 
and absence of bias. Decreased reliability leads to 
decreased credibility when the SBA reviews the performance 
evaluations. 
e. Completeness 
Incomplete measures may motivate people to per- 
form only a portion of the behavior needed for organization- 
al effectiveness. Standard Form 1420 allows contractors to 
be rated on: 
- Quality of Work 
- Timely Performance 


- Effectiveness of Management 
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- Compliance With Labor Standards 
- Compliance With Safety Standards 

While these broad categories cover the range of 
important elements in contractor performance, they lack 
specificity, and therefore, fail to provide an adequate 
assessment of contractor performance. Contract specialists 
meeor 15), AROICC's (#1 of 62), and supervisory civil 
engineers (2 of 7) said the evaluations would be more 
valuable if more ee evaluation items were used. 
Though these numbers don't represent a majority of the 
survey populations, they are significant because the 
responses were made to an open-ended question. Within the 
context of ene model, the OICC as decision maker would 
receive more information if more specific items were 
evaluated. 

f. Requisite Variety 

The OICC as decision maker must look at a con- 
tractor's record of past performance and decide whether or 
not to give him another award. As currently used, a 
marginal contractor, a satisfactory contractor, and a very 
good contractor all receive a satisfactory rating. The 
system does not allow a distinction between those levels of 
performance. Contract specialists (7 of 15) and AROICC's 
(11 of 62) said there should be additional ratings (1.ée., 
marginal and highly satisfactory). These were also 


responses to an open-ended question. These extra categories 


Sl 


would provide OICC's more precise information about a con- 
tractor's past performance. 
g. Valence 

To what degree are the measurements tied to the 
organizational reward system? Of 20 offices surveyed, 12 
withheld no contract awards due to unsatisfactory contractor 
past performance. Only 3 of 20 withheld more than 1 
GONECACE: 22 of 32 contractors responding had never seen a 
performance evaluation before. 13 of 62 AROICC's commented 
(in response to an open-ended question) that the evaluations 
were not useful because unsatisfactory contractors continue 
to receive awards. 3 of 62 said NAVFAC needs a meaningful 
noua system to recognize its outstanding contractors. 
Clearly there is an absence of valence within the evaluation 


system. 


B. USE ILII--CONTROLLING ONGOING CONTRACTOR PERFORMANCE 
1. Review 
NAVFAC's use of contractor performance evaluations 
to control ongoing contractor performance is analyzed below. 
This model reproduced from Chapter III depicts the control 


system: 
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CONTROL SYSTEM 


AROICC 





observes | / evaluates 
performance | / performance 


: i 
a, 
[ CONTRACTOR | 


The FAR, DFARS, and NAVFAC P-68 do not acknowledge 


2. Analysis 


this potential use for contractor performance evaluations. 
2 of the 5 EFD's surveyed (Pacific and Atlantic) do offer 
guidance for issuing interim evalUeeene: as feedback to con- 
tractors. When analyzed from the viewpoint of accepted con- 
trol theory, NAVFAC'S current use of the contractor 
performance evaluation system to control ongoing contractor 
performance is weak in these areas. 
a. Relevance 

Looking backward, relevance implies feedback 
value. NAVFAC notifies a contractor of his performance 
evaluation only if his performance was unsatisfactory (6.4% 
of the time). That notification is made after the contract 
is already completed. To the contractor as decision maker 
(deciding how to behave), that information is not relevant 


(because it is not timely). The behavior in question is 


aye: 


completed and cannot be changed. Within the context of this 
model the performance evaluations are irrelevant. 

Interim evaluations would provide contractors 
relevant information. 10 of 15 AROICC's who've used interim 
evaluations before observed a noticeable improvement in 
contractor performance. 13 of 19 contractors said thevad 
like to receive interim evaluations because they provide 
feedback on their performance. 

b. Usefulness For Decision Making 

18 of 19 contractors said they'd like to receive 
copies of their final performance evaluation. The contrac- 
tors also placed a fairly high value on their evaluations 
they've received in the past (average 7.9 on a scale of 1 to 
1L0)n: Although the final evaluations would not provide 
timely information about their performance on the ongoing 
contract, it would give the contractors feedback necessary 
to adjust their performances on future contracts. 

c. Understandability 

Information cannot be useful to decision makers 
who cannot understand it. The contractors as decision 
makers must be able to understand the evaluations they 
receive. The lack of standards, training, and a policy 
statement to contractors about the uses of evaluations 


decreases their understandability. 
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dad. Requisite Variety 
Ties tractor ws fa problem: within the context of 
this model also. "Tndividuals tend to achieve easy goals 
but not exceed them." (Lawler and Rhode, 1976, p. 42) 
Since NAVFAC rates both marginal and very good contractors 
as satisfactory, marginal performance has become the "easy 
Seal" of contractors. With more rating categories, the 
marginal performers can be identified, and all contractors 
can strive to become exceptional or outstanding. 
e. Completeness 
Recall that incomplete measures motivate people 
to perform only a portion of the behavior needed for organi- 
zational effectiveness. More specific rating items will 
provide contractors with more specific, and thus’ more 
complete feedback. The contractors will then be able to 
focus their efforts on all the behaviors needed to complete 
the contracts according to NAVFAC's standards. 
Mee CoOntcr ol labia lity 
The contractor has control of his own behavior. 
However, if he does not know what behavior is acceptable 
because he lacks feedback, the evaluation he receives is not 
completely within his control. With interim evaluations the 
contractor would have some very definite feedback mid-way 
through his performance, and could modify his behavior 


accordingly to improve his final evaluation. 
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C. USE III--CONTROLLING AROICC PERFORMANCE 
1. Review 

Recall in this model that the AROICC uses informa- 
tion from contractors' past performance evaluations to pre- 
dict future contractor behavior. That future behavior can 
then be anticipated and better managed by the AROICC. 
Within this model the OICC places management emphasis on the 
completion and use of contractor performance evaluations by 
including them as a criteria for the AROICC's personal per- 
formance rating. This model reproduced from Chapter III 


depicts the control system. 


CONTROL SYSTEM 







OLee 






DETECTOR——— EVALUATOR | Oy pp Oa KO) 54 


Performance 
Rating 


Observed 
Evaluation 


Preparation 
and Use 


AROPEC Olcc 


IDETECTOR- > EVALUATOR pre hen 


f 


| CONTRACTOR 
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2. Analysis 
The data shows that only 38% of the AROICC's have 
referred to a contractor's past performance evaluations 
before the start of a new contract. 40% of AROICC's said 
evaluations are stressed by management only when they are 
overdue or unsatisfactory. 45% of contract specialists said 
management placed less than a fair amount of stress on 
evaluations. When analyzed from the viewpoint of accepted 
control theory, NAVFAC'S current use of the contractor 
performance evaluations to control AROICC performance is 
weak in the following areas. 
a. Usefulness For Decision Making 

The evaluations can be used for many purposes 
within the context of this model. Recall that 60% of the 
AROICC's surveyed did not know where to find copies of the 
evaluations. 61% of the AROICC's never refer to contrac- 
tors' past evaluations prior to administering a contract. 
That knowledge about a contractor's history of performance 
would be useful when developing a strategy for administer- 
ing and inspecting a contract. Additional attention to weak 
areas of performance may avoid repeating past problems. 
These comments by Schmode, the supervisory civil engineer at 
New London, CT provide one view on this subject (see 

Appendix I for a copy of Schmode's complete memorandum) : 
The specific report required by reference (a) is fine for 
the purpose intended but does not provide us with the 


information that would prove useful in fulfilling our 
administrative responsibilities for follow-on contracts 
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with the same contractor. The five performance elements 
identified in the reference (a) required report (SF 1420) 
are much too general and the wide dispersion between 
"satisfactory" and "unsatisfactory" leave much to be 
desired. Basically, we need to know who our marginal 
contractors are, who our satisfactory contractors are that 
have some marginal or unsatisfactory traits. If we know 
in advance of the specific areas in which our contractors 
have proven themselves to be less than satisfactory, we 
can give our attention to those specific areas including 
emphasis during the Preconstruction Conferences. sie) 
short, we need a system that can be accurate, objective, 
consistent, require minimum effort, and can be cumulative. 
(Schmode, Bocorwe. 1) 


The remainder of Schmode's memorandum describes such a 
system. 
b. Reliability 
26% of the AROICC's surveyed did not know their 


EFD's had an instruction covering contractor performance 


evaluations. Only 38% of AROICC's had ever received 
training on how to complete an evaluation. These facts 


indicate there may be a lack of consistency, stability, and 
dependability in the evaluations prepared by those AROICC's. 
If oIcc's use the contractor evaluations as a rating criter- 
la of the AROICC, then the AROICC's will be motivated to 
prepare higher quality evaluations. These higher quality, 
more reliable evaluations will consequently improve the use- 
fulness of the evaluations for the other two purposes previ- 
ously discussed. 
c. Reguisite Variety and Completeness 

Increased management emphasis on evaluation 

preparation will cause AROICC's to evaluate contractors more 


thoroughly on all elements of their performance. The 


Sc 


resultant evaluations will improve their utility for their 
other two uses. 
d. Valence 

When asked to rate the evaluations's effective- 
ness for improving contractor performance a majority of the 
AROICC's said the evaluations were not effective (trimmed 
average 2.96 on a scale of 1 (not effective) to 10 (very 
effective)). 40% of the AROICC's said management emphasizes 
the evaluations less than occasionally. Poor G2) AROLEe's 
Said the evaluations are not effective because contractors 
rated unsatisfactory continue to get awards. There is a 
general impression that the evaluations do not make any 
difference. This low valence level adversely affects the 
quality of evaluations prepared and the use of evaluations 


by the AROICC's. 


D. SUMMARY 

This chapter analyzed each of the three possible uses 
for contractor performance evaluations from the viewpoint of 
accepted control and measurement theory. The next chapter 
uses that analysis as a basis for some conclusions and 
recommendations about NAVFAC's current use of construction 


contractor performance evaluations. 
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VI. CONCLUSTONS AND RECCIMMENEAELONS 


The previous five chapters included the background, 
theory, data, and analysis about NAVFAC's current use of 
contractor performance evaluations. This chapter draws some 
conclusions about that use and makes recommendations on how 


NAVFAC can improve its use of the evaluations. 


A. CONCLUSIONS 
After reviewing the information in previous chapters, 
the following conclusions are made about NAVFAC's current 
use of contractor performance evaluations: 
1. NAVFAC lacks standards of performance which clearly 
describe the difference between unsatisfactory, satis- 


factory, and outstanding performance. 


2. The constructions contractors are not generally aware 
of the performance evaluation system. 


3. The evaluations are not used to provide contractors 
feedback about their performances. 


4, The evaluations are not timely, because they are 
written after the performance is completed. 


5. AROICC's are not well trained in completing the evalu- 
ations. The lack of training adversely affects the 
reliability of the evaluations. 


6. Management emphasis on the evaluation process could be 
improved. 


7. There is no well publicized incentive for contractors 
to provide outstanding performance. 


8. The evaluations are under-utilized for each of the 
three possible uses. 
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10. 


B. 


The Standard Form 1420 is not specific enough in iden- 
tifying the elements of contractor performance to be 
evaluated. 


The data base of information from the evaluations is 
inadequate. 


SHORT-TERM RECOMMENDATIONS 


These recommendations for improving NAVFAC's contractor 


performance evaluation system could be implemented quickly. 


a. 


Policy Statement--NAVFAC should issue a policy state- 
ment describing: 


a. The uses of the Conemaccor performance 
evaluations. 


b. The standards of performance required for satis- 
factory, outstanding, and unsatisfactory perform- 
ance ratings. 


c. An explanation of the evaluation elements. 


This policy statement would increase contractor 


awareness of the evaluation system and improve the consis- 
tency, completeness, reliability, and usefulness of the 


evaluations. A draft policy statement is included in 
Appendix J. 
2. Interim Evaluations--To provide contractors timely 


feedback about their contract performance, NAVFAC 
should require field offices to issue interim evalua- 


tions. The Standard Form 1420 can be used with a 
cover letter explaining the purpose of the interim 
evaluations. Some field offices may view this 


requirement as excessive due to their already heavy 
administrative workload. The time spent preparing the 
interim evaluations should be offset by time saved 
when improved contractor performance decreases the 
administrative burden. The interim evaluations are 
also useful and should be required when the AROICC on 
a contract is replaced. 


Contractor Notification--NAVFAC should provide all 
contractors a copy of their performance evaluations. 
The feedback value of these evaluations should not be 
wasted. Knowing the evaluations are sent to the con- 
tractors should increase the AROICC's objectivity when 
preparing them. 
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4. Training--Field offices should train AROICC's on how 
to complete evaluations. The AROICC's should also 
learn the purposes and standards for the evaluations. 
This training can be conducted at the field level 
using the policy statement as a guide. The training 
will improve the consistency, completeness, and relia- 
bility of the performance evaluation process. 


5. Management Emphasis--Management personnel should rate 
their AROICC's on how well they prepare and use the 
performance evaluations. This increased management 
emphasis on the evaluations will improve their use as 
a source of information about prospective contractor 
performance histories. AROICC's will take more care 
in preparing contractor evaluations if they know their 
personal performance rating is at stake. 


6. Reward  System--All contractors rated outstanding 
should receive an award for their work. The awards 
should be meaningful and immediate. The contractors 
should know in advance what award is given for 
outstanding per-formance. In the short term, types of 
awards that can be given include: 


a. Special command recognition in the form of news- 
paper articles, special ribbon cuttings, letter 
of commendation, etc. 


b. Recommendation to other clients about the con- 
tractors outstanding work. 


7. Publicize the System--When performance evaluations are 
used to deny an unsatisfactory contractor a contract 
award NAVFAC should publicize it. Too many people 
neglect the evaluation system because they feel it 
doesn't work. 


8. Justify Satisfactory Ratings--A narrative should be 
required for satisfactory evaluations also. AROICC's 


should cite a contractor's strong and weak points even 
if the rating is satisfactory. That information is 
useful for predicting performance on future contracts. 
Not requiring a narrative on satisfactory ratings 
gives AROICC's pressed for time a quick means of com- 
pleting the evaluation requirement. 


Ce LONG-TERM RECOMMENDATIONS 
1. Modify the SF 1420--The current form does not evaluate 


contractors in enough specific areas. The form needs 
more rating categories to differentiate the marginal 
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Peonmenemcdeistactory and highly satisfactory contrac- 
tors. A draft modification of the form is included as 
Appendix K. 


Rewards--In the long term NAVFAC should be able to 
develop some tangible and meaningful reward for 
contractors who perform outstanding work. The con- 
tractor community could be consulted for suggestions. 


Improve the Data Base--Manual record keeping of con- 
tractor performance information is inadequate. Very 
few of the contract specialists surveyed use the EFD 
files of contractor performance evaluations. Aeon 
puterized data base would be used (6 of 7 supervisory 
civil engineers said they would use it). The data 
base should include: 


a. A summary of each contractor's evaluation ratings 


b. One line statements of contractor strengths and 
deficiencies. 


The data base should be accessible to field offices 
without significant assistance from EFD personnel. 
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APPENDIX A 


DEFINITIONS 


AROICC--Assistant Resident Officer-in-Charge of Construc- 
tion. Within the NAVFAC system this term refers to a Civil 
Engineer Corps Officer who directly administers construction 
contracts. Within the context of this study the term AROICC 
refers to both military and civilian personnel who directly 
administer construction contracts. 


Supervisory Civil Engineer--The senior civilian manager of a 
contracting field office. A senior Civil Engineer Corps 
officer is normally also assigned to manage in most NAVFAC 
field offices. 


Contract Specialist--Within the context of this study a 
contract specialist works in field offices: preparing con- 
tracts for advertisement and award, doing pre-award surveys, 
resolving contractual problems, and otherwise supporting the 
contracting efforts. 


OIcc--Officer-in-Charge of Construction. Usually a senior 
Civil Engineer Corps Officer with delegated contracting 
AUienOr iseye By virtue of that authority he is responsible 
for the success of all contracting actions he issues. The 
OICC usually prepares personnel performance evaluations on 
AROICC's within his jurisdiction. The OICC also can recom- 
mend denying a contractor an award based on his determina- 
tion that the contractor is not responsible. 
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Peek NOL 5 


STANDARD FORM 1420 


FOR OFFICIAL USE ONLY 
(WHEN COMPLETED} 
1. CONTRACT NUMBER 


PERFORMANCE EVALUATION — CONSTRUCTION CONTRACTS 


PART | — GENERAL CONTRACT DATA 
2. CONTRACTOR (Name, address and ZIP code} 






8. NEGOTIATEO 








CONTRACT FIRM 
(Chacha) "| FIXED OTHER 
fe cee ene once ‘Specify ) 






4. COMPL eX! OF WORK 


& DIFFICULT 





$. DESCRIPTION ANDO LOCATION OF WORK 
















C. LIQUIDATED DAMAGES (| O. NET AMOUNT PAID 
ASSESSEOQ CONTAACTOA 


EB 


O. DATE WORK ACCEPTEO 


8. TOTAL AMOUNT O 
MODIFICATION 










[de AMOUNT OF BA 
DATA | 


$ 


Cc. REVISED CONTRACT 
COMPLETION DATE 






8. ORIGINAL CONTRACT 
COMPLETION DATE 


PART tl — PERFORMANCE EVALUATION OF CONTRACT (Cheek soproonace box) 


9. PERFORMANCE ELEMENTS OUTSTANDING | SATISFACTORY [UNSATISFACTORY 


Se = 
ae = te ei A hh ip 


A. QUALITY OF WORK 


8. TIMELY PEAFOAMANGE li f= 


€_ EFFECTIVENESS OF MANAGEMENT a a 
0. COMPLIANCE WITH LABOR STANDARDS En Se a 


E. COMPLIANCE WITH SAFETY STANDARDS 
10. OVERALL EVALUATION , 
DUTSTANDING (zplein in item 13. on revere! C SATISFACTORY UNSATISFACTORY (Explain in [tem 14, on reveres) 


11 EVALUATED BY 


A. OAGANIZATION (Type or print} 


8. NAME ANDO TITLE (Type or pnant) C. SIGNATURE 





12. EVALUATION REVIEWED BY 


? AB. ORGANIZATION (Type or pant) 


8. NAME AND TITLE (Type or print) 








FOR OFFICIAL USE ONLY STANDARD FORM 1420 (10-83) 
Prescribed by GSA 


{WHEN COMPLETED] FAR (45 CFA) $3.236-1(0) 
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FOR OFFICIAL USE ONLY 
(WHEN COMPLETED) 


ee RE Sy Pps sis SSS SPSS ay ga 

13. REMARKS ON OU TSTANOING PERFORMANCE AS INOICATEQ dy THE CONTRAC TOQ'S PERFORMANCE ON THIS CONTHOEaCT 1F ©~OU CON. 
stOER FHE CONTRACTOR TO BE OUTSTANOING, SET FORTH FACTUAL OQATA SUPEQRKTING HIS USBSERYASTION theSe DATA MUST BE IN 
SUFFICIENT OETAIL TO ASSIST CONTRACTING OFFICERS IN SELECTING CONTRACTORS THAT HAVE OEMONSTHAI EU OUTSTANDING 
QUALITY OF WORK ANDO RELIABILITY. (Continua on separate snect, if needadt j 


~4. ob ie 


—— oe eee eee. 





14 EAPLANATION OF UNSATISFACTORY EVALUATION - FOR EACH UNSATISFACTORY ELEMENT, PROVIOE FACTS CONCERNING SPECIFIC 
EVENTS OR ACTIONS TO JUSTIFY THE EVALUATION (eg. extent of Gouermment tnepe: tw rmquued, rewort required. subcontracting. cooperation 
cfc: tre tar quality of wortmen und adequacy of equipment) THESE OATA MUST BE IN SUFFICIENT OF TALL [O AddsisT CONTHACTING 
Or FILLERS IM CETERMINING THE CONTRACTOR'S RESPONSIBILITY. (Conanue on seperet sreet, if needed.) 





FOR OFFICIAL USE ONLY STANDARD FORM 1420 BACK (10-83) 


1983 0 - - ui 
) 381-526 (9007) (WHEN COMPLETED) 
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APPENDIX C 


AROICC SURVEY 


1. Approximately how many satisfactory, outstanding, and 
unsatisfactory contractor performance evaluations have you 
written during your career as a contract administrator? 


Satisfactory Outstanding Unsatisfactory 


2. Does your EFD have an instruction governing contractor 
performance evaluations? YES/NO 


If yes, how much help does it provide when you are 
filling out an evaluation? (CIRCLE ONE) 


A BIG HELP SOME BE GE /NS SEARLE U a, TBO Sy NO HELP 


3. Listed below are some possible uses of performance 
evaluations. Please rank them in order from (1) meaning the 
most important use, to (5) meaning the least important use. 
(If you add a use in the other spot, rank from 1 to 6) 


Use to Provide Feedback to the Contractors 

Use to Support Findings of Non-Responsibility, and thus 
Deny Contractors an Award 

Used by Contract Specialists During the Pre-Award Survey 
Use to Give AROICC's Information About a Prospective 
Contractor 

Use to Satisfy the FAR Requirement to Complete an 
Evaluation 

OTHER (Please Specify) 


4. Have you ever received any training on how to complete 
the performance evaluation form? YES/NO 


If yes, which of these statements most accurately 
describes the level of your training? (CHECK ONE) 


Formal Classroom Training 
Briefed during ROICC Staff Meeting 
OJT by Contract Specialist or another AROICC 
Given a Copy of an old Evaluation as a Guide 
Other (Please Specify) 
5. Do you ever review a contractor's record of past per- 
formance prior to the start of a new contract? (YES/NO) 


Ww / 


If yes, approximately how often do you do so? (CIRGCEE 
ONE) 


Always 15% OL 50% of 252 0n 5% of 
the time the time the time they tane 


6. Do you know where you could find a file of all of a 
particular contractor's past performance evaluations? 
YES/NO Where? 


7. On a scale of 1 to 10, how effective do you think the 
existing performance evaluation system is at improving the 
overall performance by NAVFAC’ contractors? (CIRCLE A 
NUMBER) 


it c 3 4 5 6 7, 8 9 10 
Not Effective Very Effective 


8. Have you ever written an interim evaluation on a 
contractor mid-way through a project? YES/NO 


If yes, what effect does the interim evaluation have on 
the contractor's performance? (CIRCLE ONE) 


Improvement Some Improvement Little Improvement No Change 


9. Do you think interim evaluations would help you to 
positively influence a contractor's behavior? YES/NO 


Why? 


10. Which of these statements most accurately describes how 
much emphasis your office management places on the perform- 
ance evaluations? (CHECK ONE) 


Evaluations Are Stressed Frequently 

Evaluations Are Stressed Occasionally 

Evaluations Are Stressed When A Contractor's Performance 
is UNSAT 

Evaluations Are Stressed When They're Overdue to the EFD 
Evaluations Are Never Stressed 


11. What comments do you have about the way we evaluate our 
contractors? Please include any suggestions you have for 
improving the evaluation process. (Use back of page if you 
need more room.) 
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APPENDIX D 


CONTRACTOR SURVEY 


1. On a scale of 1 to 10, how much value do you assign to 
the performance evaluation rating you receive on a project? 
(CIRCLE ONE) 


i: 2 3 4 S 6 7, 8 9 10 
No Value High Value 


2. Of those you've received from NAVFAC, how objective and 
well documented were your performance evaluations? CGH EG 
ONE) 


Very Objective and Well Documented 
Pretty Objective and Well Documented 
Not very Objective or Well Documented 
Subjective. No Documentation 








3. Of the performance evaluations you've received from 
NAVFAC, did they provide you with useful feedback about your 
performance? YES/NO 


If YES, how was the feedback useful? 


4. An interim evaluation is one provided mid-way through a 
project. How useful do you think interim evaluations would 
be in providing you feedback about your performance on a 
@emeract: (CHECK ONE) 


VERY USEFUL 
PAEREY USEFUL 
USEFUL 

Ore DELILE USE 
NOt SOSErUL 








5. Would you like to see NAVFAC issue interim performance 
evaluations on its contracts? YES/NO WHY? 
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6. Do you have a system for evaluating your subcontractors? 
YES/NO 


If yes, could you please describe your system or enclose 
some information about your system (1i.e., rating forms) ? 


7. Would you find it useful to receive a copy of your 
performance evaluation for each contract you perform? 
YES/NO 


Why? 


8. What comments do you have about the way NAVFAC evaluates 
its contractors? Please include any suggestions you have 
for improving the evaluation process. 
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APPENDIX E 
SONTRAGC Tm SPECTALISMSSURVEY 
1. What %@ of the time do you refer to a contractor's past 


performance evaluations to assist you during the Pre-Award 
Survey process? 


oe 


0 10% 30% 50% 70% 90% 100% 


On what types of awards (i.e., unknown contractor, MCON job, 
etc.) 


2. What %@ of the time do you discuss the pre-award survey 
results with the AROICC assigned to the project? 


oe 


0 10% 25% 50% 70% 90% 100% 


Under what circumstances do you do so? 


3. What source/sources do you use to find copies of a 
contractor's past performance evaluations? (i.e., EFD file, 
momec file) 


4. About what % of the time do you use your EFD's past 
performance evaluation files when doing a Pre-Award survey? 


0% 10% 25% 50% 715% 90% 100% 
Under what circumstances? (1i.e., MCON job) 


5. How would you rate the quality of the performance evalu- 
ations you've seen in your office? 


Do you think they'd stand up in court in support of a non- 
responsibility determination? 


6. How much emphasis is placed on the evaluation process 
within your office? 


A LOT 

A FAIR AMOUNT 
SOME 

A LITTLE 

NONE 








71 


7. In the past year, how many contract awards have you 
Withheld from the apparent low bidder because of his record 
of past performance? 

Was it easy to do? Y/N Why not? 


Would it have helped if you had 3 or 4 well-documented 
unsats to reference in your request for award denial? Y/N 


8. What comments or suggestions for improving the evalua- 
tion process do you have? 


Can you give me the names of 


i. Supyeivi How long been on job 
2. Civilian Contract Administrator 
How long 
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APPENDIZ F 


PAG@ylemeiy fo lO eOnod PUCT LON CONTRACTOR 
Poe mOnMAnee EVALUATION REPORT--SUPPLE- 
MENTAL INFORMATION SHEET 

a ease 4330.4 


CONSTRUCTION CONTRACTOR PERFORMANCE EVALUATION REPORT 
SUPPLEMENTAL INFORMATION SHEET 


Gode: O = Outstanding E = Excellent S = Satisfactory 
Marginally Satisfactory U = Unsatisfactory 


<< 
i) 


bom Cc © oO R 0 & os Mf U 
1. Ability to prepare and timeliness in submission 
of schedule of prices, bill of materials and 
BRoeress ChartS/networks .« « cbse 2% « 6 + 8 ole 
2. Ability to submit shop drawings, brochures, 
and catalog cuts in accordance with contract 


ROCUMNGeEMONCS 6 6 5 « « s 6 « 8 « 6 8 8 8 6 + te le 


3. Ability to follow up and control equipment and 
Peipemecerequirements .. . « « » s « © » » © 6 « «6 * 


4. Ability to deliver materials to jobsite on time. 


5. Quality of Management: a. Field Management... 
b. Office Management. . 


Pmmoualveveor, WOrkmaniship « « . s « s « 6 6 « » « « 


pameebimely completion of contract . . « « « « « » « 


9. Capability to provide adequate equipment and 
Peromseeas Meeded . « « 2 «6 6 st et te el le tlt we ltl lt ll 


10. Capability to regain lost time when behind 
schedule e e e e ° e e es e e e e e ° e e e e e ° ° e 


mie "Gd paclty GO secure adequate labor... ... « 
ewe Compliance with safety standards .... . =. 


13. Ability to pay subcontractors, material and 
personnel e e e e e e e e e e e e e e e e e e a e ° 


14. Timely response to Government requests ...- .- 


| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
7. Compliance with security requirements ..... | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
ioemovendllepertormance rating . « . + « 6. «= - | 


—_— rr ms rm re ee ees ees we ee eee 
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APPENDIX H 
Sibi toOkRwecliVil ENGINEER SURVEY 
1. How well does your EFD instruction governing contractor 


performance evaluations help the AROICC's when they're 
filling out the evaluations? 


2. What changes would you suggest to improve the usefulness 
of the instruction? 


3. Do you provide your AROICC's any training on how to 
complete the performance evaluation form? YES/NO (Circle 
One) 


What kind of training? 


4. Does your office keep separate performance evaluation 
files on each contractor? YES/NO (Circle One) 


5. On a scale of 1 to 10, how effective do you think the 
existing performance evaluation. system is at improving the 


overall performance by NAVFAC contractors? (CIRCLE A 
NUMBER) 

1 2 3 4 5 6 7 8 9 16. 
Not Effective Very Effective 
6. Does your office write interim evaluations on contrac- 


tors mid-way through a project? YES/NO 
If yes, what do the interim evaluations usually have on 
the contractor's performance? 

7. Do you think NAVFAC should issue interim evaluations on 


all of its construction contracts? YES/NO 


Why? 


15 


8. What do you think of the form we use for contractor 
performance evaluations (SF 1420)? Would you make any 
changes to the form, and if so what changes? 


9. If you had access uSing the office computer to a file 
listing all of a contractor's past performance evaluations, 
would you use it to get a feel for how a prospective con- 
tractor of yours usually performs? YES/NO 


10. Do you think we should give our contractors a copy of 
their performance evaluations at the end of a contract? 
YES/NO Why or why not? 





11. What comments do you have about the way we evaluate our 
contractors? Please include any suggestions you have for 
improving the evaluation process. (Use back of page if you 
need more room) 
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Poe eNO ix i 


SCHMODE M=MORANDUM ON CONTRACTOR 
PeeeOn ewes PaPORe (CPR) 


4, DEPARTMENT OF THE NAVY 
persia Aine es Memorandum 


OATE: 26 Nov 85 
FROM: Code R-1 


TO: Distribution List 


$UBJ: CONTRACTOR PERFORMANCE REPORT (CPR) A 


Ref: (a) NORTHNAVFACENGCOMINST 4335.5 Subj: Performance Evaluations of 
Construction Contractors 


Encl: (1) Sample CPR Form 


1. We are tasked by reference (a) to evaluate the performance of our various 
construction contractors at the conclusion of their contract. The specific 
report required by reference (a) is fine for the purpose intended but does 
not provide use with the the information that would prove useful in 
fulfilling our administrative responsibilities for follow-on contracts with 
the same contractor. The five performance elements identified in the 
reference (a) required report are much too general and the wide dispersion 
between “satisfactory” and “unsatisfactory” leave much to be desired. 
Basically, we need to know who our marginal contractors -are, who our 
satisfactory contractors are that have some marginal traits and what these 
marginal or unsatisfactory traits are. If we know in advance of the specific 
areas in which our contractors have proven themselves to be less than 
satisfactory, we can give our attention to those specific areas including 
emphasis during the Preconstruction Conferences. In short, we need a system 
that can be accurate, objective, consistent, require a minimum of effort, and 
can be cumulative. 


2. Talking to a number of you, the following 12 factors Seem to cover the 
Majority of the significant elements that Should be our concern: i 


(1) Superintendent. 

(2) Safety. 

(3) Timely Completion. 

(4) Shop Drawings. 

(5S) Completion of *Punch List*. 
(6) As-Built Drawings. 

(7) Change Order Negotiations. 
(8) Cooperation. 

(9) DRIs. 

(10) Payrolls. 

(11) Quality Assurance. 

(12) Technical Capability. 


id 


A brief description of what 18 intended for each element is in order. 


Superintendent. On site when work in progress; coordinates work of 
subcontractors; expedites work; controls project, etc. 


Safety. Site cleanliness; safety meetings; enforces hard-hats; accident 
reports, etc. 


Timely Completion. Project B.O.D.‘d within current contract completion date. 


Shop Drawings. Timely submission and complete; contains KR certification 
stamp; submittal log. 


Completion of Punch List. Completion within reason after BOD; works on own 
initiative. : 


As-Built Drawings. Keeping current as job progresses; timely submission at 
BOD. 


Change Order Negotiations. Timely response to RFPs; reasonableness of cost 
breakdown; Willingness to negotiate a fair and reasonable price and time. 


Cooperation. Willingness to work with inspector and project engineer. 

DRIs. Timely submission and completeness. 

Payrolls. Timely submission without discrepancies. 

Quality Assurance. Workmanship and willing compliance with plans and specs. 


Technical Capability. Competency of contractor to do the work; ability to 
provide adequate workmen and equipment. 


3. A Simple grading system that is self-explanatory is as follows: : 
Outstanding - 5 
Above Average - 4 
Satisfactory - 3 
Marginal - 2 
Unsatisfactory - l 


It is expected that a majority of our contractors will fall in the 
*“Satisfactory” category. As previously stated, it is those contractors that 
have certain elements that fall in the ‘“°Unsatisfactory"® or ‘°Marginal’” 
category that we are primarily concerned with. At the time we complete the 
more formal Construction Contractor Performance Evaluation Report, the 
project engineer and the project inspector will collectively complete this 
new Contractor Performance Report (CPR) by assigning numerical “*grades® to 
each of the 12 elements. If a grade is either 1 or 2, a very brief 
explanation is to be provided. The combined effort of the inspector and the 
project engineer should not require much over 15 to 20 minutes. The CPR will 
be prepared for all construction contracts. The cumulative results of these 
reports will be posted in our computer and will be available for immediate 
recall for use on subsequent contracts with the contractor involved. 
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4. A sample of the CPR form to be used iS attached as enclosure (1). The 

form provides for the indication of the general classification of the type of 

project involved, e.g., electrical, painting, general, etc. Please advise if 

you have any suggestions on how to improve this reporting system or the form 

iteself. This report is for internal use only ang the information will 

neither be forwarded to Northern Division nor yven rectly to the 
¢ 


contractor involved. 4) 
as 
, 
G. W. SCHMODE 


Supvry Civil Engineer 
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APPENDIX J 


DRAriehO Lite no La LEMENE 


A. INTENT: The Federal Acquisition Regulations and the 
Department of Defense FAR Supplement direct NAVFAC to issue 
Sencractor performance evaluations on most Ot its 
construction contracts. This policy statement is intended 
(Eloy 


(1) Establish the purposes for which NAVFAC intends to 
use the contractor performance evaluations. 


(2) Provide standards of performance necessary to achieve 
the various performance ratings. 


(3) Emphasize the activities included in each evaluation 
item. 


(4) Promulgate direction for distribution of contractor 
performance evaluations. 


This statement should increase the awareness, under- 
standing, reliability, objectivity, and usefulness of the 
contractor performance evaluations. Comments about this 
policy statement should be addressed to: 

Code 022 

Naval Facilities Engineering Command 
200 Stovall Street 

Alexandria, Va 22332 


B. USES: The contractor performance evaluations will be 
used: 


(1) To support determinations of contractor responsi- 
bility during pre-award surveys. 


(2) To provide contractors feedback about their ongoing 
performance. 


83 


(3) By AROICC's to assess a contractor's strengths and 
weaknesses prior to administering a new contract. 


C. STANDARDS: NAVFAC will augment the three existing 


rating categories on the Standard Form 1420 by adding 


categories of marginal and highly satisfactory. These 


standards describe typical performances worthy of each 
rating: 


Outstanding-- The contractor should meet or exceed all 
contract requirements; display a high level of 
professionalism in all aspects of performance; 
anticipate all potential problems within the 
realm of his responsibility and implement 
effective solutions; cooperate fully (within 
the requirements of the contract) with the 
AROICC; minimize the amount of government 
administrative effort on the contract. 


Highly 

Satisfactory--The contractor should: meet or exceed all 
contract requirements; display professionalism 
in all aspects of performance; anticipate most 
potential problems within the realm of his 
responsibility and implement effective solu- 
tions most of the time; cooperate (within the 
requirements of the contract) with the AROICC; 
minimize the amount of government administra- 
tive effort on the contrace- 


Satisfactory--The contractor should: meet all contract 
requirements; display an average amount of 
professionalism; anticipate some problems 
within the realm of responsibility and imple- 
ment effective solutions some of the time; 
cooperate (within the requirements of the 
contract) most of the time; require only a 
normal amount of government contract adminis- 
trative effort on the contract. 


Marginally 

Satisfactory--The contractor should: meet most of the 
contract requirements, including those criti- 
cal to the use of the particular facility; 
display some professionalism; cooperate 
(within the requirements of the contract) 
minimally with the AROICC; react to most 
problems rather than anticipate them; require 
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an excessive amount of government administra- 
tive effort on the contract. 


Unsatisfactory--The contractor has failed to meet the mini- 
mum contract requirements even after signifi- 
cant administrative effort by the government. 
The contractor was uncooperative, failed to 
anticipate problems within the realm of his 
responsibility, and did not attempt to develop 
solutions. The contractors actions were not 
professional. 


D. EVALUATION ELEMENTS--Within the 5 broad evaluation 


elements on the SF 1420 contractors will be specifically 


rated on: 
Broad Element Specific Element 
A. Quality of Work 1. Materials Used 
2. Workmanship 
3. Quality Assurance Plan and 
Efforts 
B. Timely Performance 1. Scheduling 


2. Punch List Completion 


C. Effectiveness of 1. Effectiveness of 
Management Superintendent 
2. Cooperation 
3. Administration (1.e., sub- 


mittals, payrolls) 


D. Compliance With 1. Payrolls 
Labor Standards 2 Violations 

E. Compliance With 1. Safety Awareness and 
Safety Standards Attitude 


2. Safety Plan 
3. Safety Record 
Although these specific items may have been implicitly 
evaluated using the SF 1420, they will now be explicitly 
evaluated using this modified system. The contractors and 


AROICC's both know what areas of performance will be 
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evaluated. The general standards can be applied to each 
specific evaluation element. As an example, the following 
standards could apply to Quality of Work-Materials Used: 


Outstanding --All materials met or exceeded specific quality 
requirements. Complete and acceptable 
material submittals were made as required. 
Manufacturers recommendations and appropriate 
industry practices were followed when using 
all materials. Obvious care was taken when 
storing and handling materials. No government 
effort was needed to ensure compliance with 
materials requirements. 


Highly 

Satisfactory--All materials met or exceeded specified qual- 
ity requirements. Complete and acceptable 
material submittals were made as required. 
Care was taken when storing and handling 


materials. Manufacturers recommendations and 
appropriate industry practices were followed 
most of the time. Very little government 


effort was needed to ensure compliance with 
material requirements. 


Satisfactory--All materials met or exceeded specified qual- 
ity requirements. Complete and acceptable 
submittals were made as required. Some resub- 
mittals were needed before final material 
approval was granted. Normal care was usually 
taken when storing and handling materials. 
Manufacturers recommendations and industry 
practices were usually followed. Government 
effort was required occasionally to ensure 
compliance with material requirements. 


Marginally 

Satisfactory--All materials met specified quality require- 
ments. Complete and acceptable submittals 
were made only after several attempts. Below 
average care was taken when storing and 
handling materials. Manufacturers recommen- 


dations and appropriate industry practices 
were occasionally violated. A large amount of 
government effort and some rework was required 
Leo) ensure compliance with material 
requirements. 


Unsatisfactory--Not all materials met specified quality 
requirements. Material submittals were 
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incomplete and often unacceptable. Little 
care was taken when storing and handling 
materials. Manufacturers recommendations and 
appropriate industry practices were Not 
routinely followed. A significant amount of 
government effort and rework was required to 
bring the contractor into compliance with 
material requirements. 
E. PREPARATION AND DISTRIBUTION: When reasonable to do so, 
or when requested by a contractor, the AROICC will issue an 
interim evaluation at the mid-point of the contract 
duration. The interim evaluation is intended to provide 
contractors feedback about their performance. The contrac- 
tors final performance evaluation rating is intended to rate 
the contractors performance over the entire period and 
therefore should not depend necessarily on the interim 
evaluation ratings. 

Interim evaluations should be sent to the contractors at 
the mid-point of the contract duration. tie a COntEact 1s 
for an extended period of time, interim evaluations should 
be written at least for every 6 months of performance. 


Final performance evaluations will be sent to contractors at 


the time of final payment. 
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APPENDIX K 
DRAFT REVISION TO STANDARD FORM 1420 
These modifications are recommended for improving the 
Standard Form 1420. 
1. Items 1-8, 11 and 12 should remain unchanged, but 
the vertical space allowed for items 5, 8, 11, and 12 should 


be reduced to accommodate the changes to items 9 and 10. 


2. Change item 9 to look like this: 


9. Perform- Out- Highly Satis- Margi- Unsatis- 
ance stand- Satisf. factory Mcialay factomy 
Elements Satisf. 


A. Quality of Work 
1. Materials 
2. Workmanship 
3. QO/7A plan 


B. Timely Performance 
1. Scheduling 
2. Punch List 


C. Effectiveness of Mgmt. 


1. Superintendence 
2. Cooperation 
3. Administration 


D. Compliance w/Labor 
1. Payrolls 
2. Violations 


E. Compliance w/Safety 
1. Awareness/Attitude 
2. Safety plan 
3. Safety record 


oe Combine items 13 and 14 into one explanation and 


justification section. Require narrative supporting all 
ratings, not just outstandings or unsatisfactories. 
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